Monte Carlo Feature Selection and Interdependency Discovery in Supervised Classification

Applications of machine learning techniques in Life Sciences are the main applications forcing a paradigm shift in the way these techniques are used. Rather than obtaining the best possible supervised classifier, the Life Scientist needs to know which features contribute best to classifying observations into distinct classes and what are the interdependencies between the features. To this end we significantly extend our earlier work [Dramiński et al. (2008)] that introduced an effective and reliable method for ranking features according to their importance for classification. We begin with adding a method for finding a cut-off between informative and non-informative features and then continue with a development of a methodology and an implementation of a procedure for determining interdependencies between informative features. The reliability of our approach rests on multiple construction of tree classifiers. Essentially, each classifier is trained on a randomly chosen subset of the original data using only a fraction of all of the observed features. This approach is conceptually simple yet computer-intensive. The methodology is validated on a large and difficult task of modelling HIV-1 reverse transcriptase resistance to drugs which is a good example of the aforementioned paradigm shift. We construct a classifier but of the main interest is the identification of mutation points (i.e. features) and their combinations that model drug resistance. Michał Dramiński and Jacek Koronacki Institute of Computer Science, Polish Acad. Sci., Ordona 21, Warsaw, Poland, e-mail: Michal. [email protected],[email protected] Marcin Kierczak The Linnaeus Centre for Bioinformatics, Uppsala University and The Swedish University of Agricultural Sciences, Box 758, Uppsala, Sweden, e-mail: [email protected] Jan Komorowski The Linnaeus Centre for Bioinformatics, Uppsala University and The Swedish University of Agricultural Sciences, Box 758, Uppsala, Sweden Interdisciplinary Centre for Mathematical and Computer Modelling, Warsaw University, Poland, e-mail: [email protected] 1 These authors contributed equally.